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REMARKS AND ARGUMENTS 

A petition of time for a three month extension is submitted herewith under 
separate cover. 

The first paragraph of the applications, page 1 has been amended to add 
application serial numbers and/or patent numbers tor other applications incorporated 
by reference, as requested by the Examiner. 

Claim 15 has been amended to replace the 'invention' with the 'information' to 
correct a typographical error, and thereby overcome the §11 2 rejection. 

Examiner states applicants' arguments with respect to claims 1-8 and 10-17 
are moot in view of new grounds. The Examiner has rejected claim 1-B, 10-17 under 
35USC 103(a) as unpatentable over Focsaneanu In view of Gerszberg. 

The Examiner argues that Focsaneanu discloses only two modal forms, i.e. 
audio and visual, and does not disclose the tactile modal form, and that Gerszberg 
discloses a network server platform for Internet, Java Server, and video application 
server comprising the following features: providing multimodal user interfaces 
capable of supporting Input and output in one or more of audio, visual and tactile 
modal forms; and passing information in audio, visual and tactile form, and argues 
that it would have been obvious to one of ordinary skill in the art to modify the 
network of Focsaneanu by using the features as taught by Gerszberg et al. in order 
to improve the capabilities of the customer premises equipment and lowering overall 
system costs to the customer by providing "competitive service alternatives" (ref. col 
2, lines 30-32 of Gerszberg). 

In contrast, the current application describes a server for handling multimodal 
information and an aspect of invention described in the present application is 
directed to managing a multimodal user interface having modal sensltitivtty to audio, 
visual and tactile modes of interaction of a user, typically a human user, with an 
multimodal user interface for a communications terminal. By providing capability for 
a user to interact with a terminal using one or more modal forms of user input and/or 
output, and resolving conflicts between different modal forms of user input/output 
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that may be used simultaneously, problems for a user interacting with multimodal 
user interfaces are reduced. 

As noted for example, at page 4, fine 30 of the present application, 
preferably, the service controller is operable to receive input in different modes 
simultaneously from the same user, to resolve any conflicts, and determine an 
intention of the user based on the inputs. Thus the service controller facilitates use of 
more than one modal forms of input and output. 

Amendments to main claims 1 , 1 2, 14, 1 5 and 16 have been made to add 
feature clarifying, e.g. that the service controller or associated method, is operable Jo 
receive input in different modal forms simultaneously from a user interaction at the 
multimodal interface, and to resolve conflicts a nd thereby distinguish more clearly 
over the cited references. 

To assist in understanding distinctions over the cited art, referring to U.S. 
patent application 6,012,030 {ser. No. 09/062,970), which was incorporated by 
reference within the current application, this related application describes at col. 2 to 
col. 3, cited below, some of the problems users may have trying to use particular 
input/output modalities in existing systems; 

While interfaces for communications devices and computer systems are 
becoming Increasingly able to accept input and provide output through 
various sensory modalities, existing systems and devices present some 
problems when the user tries to use particular input/output modalities 
according to the task at hand. 

In using such an interface, for example, a user might request an item using 
speech, and then be presented with a list of choices on the screen, that 
requires some scrolling to access the relevant section of the list. At this point 
the user may choose to touch the scroll control and then touch an item on the 
list that they require- 
Ideally the user wants to smoothly transition from one type of inputyoutput 
modality to another, e.g. from a primarfly speech input/output to a graphical 
and touch control structure. However there are problems with providing this 
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transition in practice because there is an intrinsic conflict between speech 
interaction and graphical interaction styles. 

Current graphical interfaces are directed through a task by a user. Nothing 
happens unless a user dicks on a screen based object or types from a 
keyboard. The user maintains control of the interaction, and can pause and 
restart the task at any time. 

In contrast, speech interfaces tend to direct a user through a task: The user 
initiates the interaction, and thereafter the speech recognizer prompts the 
user for a response, i.e. asks the user to repeat a name, etc. and expects an 
almost immediate input. As mentioned above, speech recognizers for 
communications devices typically operate within a limited time window, 
usually within a few seconds after a speech prompt. Thus, the timing of the 
listening window of speech recognizer controls the requirement for the user 
to respond, to avoid an error or reprompting. Users often report feeling 
rushed when prompted to respond immediately after a beep or other speech 
prompt. 

For, example, a user may choose or be prompted to use one or more 
modalities, and the present Invention provides for management of user interface for 
input or output using an appropriate one of several complementary modalities 

Thus, the invention described in the present application relates to problems 
of managing the user interface at an application level. In one embodiment, there is 
described Java web server implementation providing enhanced sen/lets called 
multimedia servlets, eg. voice enabled web server applications. As noted in the 
application, aspects of the invention enable multimodal interfaces to be provided 
using thin clients. 

The invention described and claimed is therefore directed to solving a 
different problem from that described in Focsaneanu and/or Gerszberg, which the 
examiner states as providing "competitive service alternatives". Consequently, 
Gerszberg does not provide what is lacking in Focsaneanu with respect to the 
invention claimed in the present invention. Indeed there would be no motivation to 
combine Gerszberg with Focsaneanu to solve the problem addressed by aspects of 
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the present invention sine© both Focsaneanu and Gerszberg relate to other aspects 
of network communications. For example, Gerszberg relates to architecture for 
differentiation of local access services, see col. 1 1 , line 37. 

Since the invention described In the present application relates to a server for 
handling Information in different modal forms for managing a multimodal user 
interface and the cited references are not directed to aspects of managing a user 
interaction with a user interface at an application level as described in the present 
application, the applicants believe that the cited references do not provide basis for a 
s.103 rejection. 

It is therefore believed that the claims, as amended, distinguish more clearly 
over the crted references taken alone or in combination. 

The applicants respectfully request reconsideration of amended claims in 
light of the arguments presented above. 



Respectfully submitted, 
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